Subject: piggy-backing pacing responses

From: [email-masked]
Date: Fri, 3 Sep 93 18:36:29 EDT

After talking with some more folks here, I am starting to get concerned
about the additional WAN traffic load caused by my proposal not allowing
piggy-backed pacing responses.  When we consider small routers that
may need to keep window sizes small, the IPMs start adding up to a
non-trivial percentage of the frames.  It's hard to say how much of
problem this is because:
  - the frames are very short and don't go through the receiver
  - the tcp/ip header overhead and processing depends on whether
    the IPM sender decides to push IPMs out (which probably provides
    much smoother pacing operation)

I'd like to have some discussion about the advisability of allowing
a pacing response to be piggy-backed if one has data ready to send in
the reverse direction at the time one would otherwise send an IPM.
We would still allow isolated responses if one wanted to send them.

Several times, Shannon has suggested that this design point prevents
deadlock problems.  I've thought of one abstract scenario for this
(Shannon please correct or add others).  It assumes a common send/rcv
buffer space.
  - Partners A and B are severely congested.  A needs a pacing rsp
    from B to send some big frames and free up receive space.  B puts
    the pacing response on a big frame which A does not have room for,
    so it can't receive the pacing response.  So A stays congested w/o
    the PR and B stays congested since A can't receive B's big buffer.

This scenario is abstract because it is frame-oriented and ignores the
fact that with a TCP transport we are running these circuits over a
stream-oriented protocol with its own windowing.  A typical implementa-
tion might have TCP receiving segments (whose size TCP controls by its
own windowing) and handing pieces of frames to DLSw receive code which
has the role of reading and finding frame boundaries.  So it may be
possible for the DLSw receive code to detect and process a piggy-backed
pacing rsp indicator without waiting to read in the rest of a large
frame.

Another possibility to eliminate possible deadlock problems is to have
a rule that implementations must send isolated responses when in severe
buffer congestion.  But I don't think we want to get into trying to
define levels of congestion in the standard.

The problem with piggy-backing is finding room in the short DLSw
header for an explicit NWS.  We can grab some available bits, or
possibly make the rule that piggy-backed PRs imply no change in
NWS from last time and you have to send an IPM to change the NWS.
(I don't like the latter idea so much).

So how do people feel about the following proposal:
- we define a PRSP bit in the short DLSw header (could be the same
  bit as RLWI to save space)
- we define 5 or so other bits that can carry a NWS (5 bits => max = 32)
  (we should probably still have a longer NWS field in the isolated
  format in case some unforeseen configurations may need a larger
  window size.)
- all nodes which do pacing must be able to receive both types of
  responses
- a node can choose to send either piggy-backed or isolated responses
  or mixed

Peter